Skip to content

Conversation

dlarocque
Copy link
Contributor

The streaming file download was failing when downloading the data connect emulator in CI only (reasons unknown). To fix the issue, we can download the file to a buffer (Uint8Array), then write it synchronously to disk.

We preferred the streaming approach because it meant we didn't have to store the entire emulator file in memory, but I found that this buffer approach uses roughly the same amount of memory (~195mB in total for the test process).

@dlarocque dlarocque requested a review from a team as a code owner October 15, 2024 14:40
Copy link

changeset-bot bot commented Oct 15, 2024

⚠️ No Changeset found

Latest commit: 8e44e2c

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link
Contributor

Vertex AI Mock Responses Check ⚠️

A newer major version of the mock responses for Vertex AI unit tests is available. update_vertexai_responses.sh should be updated to clone the latest version of the responses: v4.0

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Oct 15, 2024

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Oct 15, 2024

* downloaded emulator. Unfortunately, we can't access `this` when inside `readChunk`'s scope, since
* it's a named function expression, and does not inherit the `this` object from it's parent.
* To work around this, we wrap the fetch in a promise,
* then once it's resolved we can access `this` in a callback arrow function that *does* inherit
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a couple of other ways around this, I think, not 100% sure. One is to set like const self = this at the top level (outside of the fetch anyway) and then you call self.binaryPath when needed. That looks weird so another better looking option is that you create a function at the top level called setBinaryPath that is just like

const setBinaryPath = (binaryPath) => this.binaryPath = binaryPath;

And then I think you can call setBinaryPath inside readChunk. I'm not 100% sure but I think they should work? If they don't, never mind.

@dlarocque dlarocque closed this Oct 15, 2024
@firebase firebase locked and limited conversation to collaborators Nov 15, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants